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Summary 


Problem and Background 


The military services have high security risk billets for 
enlisted accessions that must be filled only by extremely 


trustworthy and reliable personnel. To ensure that newly 
assigned personnel are qualified from a security perspective, 
each service uses various prescreening procedures. In 


addition, for candidates successfully passing the initial 
prescreen, an intensive and costly background investigation is 
conducted by the Defense Investigative Service and adjudicated 
by the respective service. 


There has been increasing DoD concern with espionage 
incidents and other failures of security clearance policies to 
protect classified information. There is a need to examine 
and validate new procedures and approaches, including 
behavioral science tests, that could provide valid and 
cost-effective supplemental or alternative methods for 
screening personnel. 


Objective 


The objective of the present study was to examine the 
validity of an experimental DoD prescreening instrument in the 
personnel security arena. More specifically, the Armed 
Services Applicant Profile (ASAP), a background/biodata 
questionnaire, was investigated to determine if it could be 
used to predict the results of background investigations 
conducted for recruits applying for high security risk jobs. 


Approach 


Data from an ongoing ASAP validation study were used. The 
predictor variables were ASAP item responses, along with AFQT 
and years of education. The subjects consisted of 3,257 
enlisted personnel, from an original accession sample of 
55,675, who were potentially entering high security risk jobs. 
These individuals had completed the ASAP as a part of an 
ongoing DoD study. In addition, they had successfully passed 
prescreening by their respective services and had a completed 
background investigation. 


Classification of a background investigation as an issue 
case or non-issue case represented the criterion variable. 
Issue cases are significant because they often indicate that 
there may be adverse information in the person's background 
that reflects on that person's trustworthiness or reliability 
and thus on his or her qualifications to hold a high security 
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risk job. Also, issue case status has been shown to predict 
subsequent unsuitability discharge from the military service. 
Approximately 12 percent of the background investigations were 
identified as issue cases. 


The ASAP predictor score was developed uSing a weighted 
application blank method. Approximately one-half of the total 
background investigation sample was randomly assigned to the 
scoring-key development group while the remainder was used as a 
holdout group for cross-validation purposes. 


Results 


The ASAP scoring key correlated significantly with the 
issue case criterion measure (r = .36). In addition, the 
validity of the ASAP key held up well across different 
Subgroups (i.e., individual services, white/minority, 
male/female). Results indicated that almost 24 percent of the 
those individuals having the lowest 20 percent of the ASAP 
scores had issue case background investigations. On the other 
hand, among the highest 20 percent of ASAP scorers, only 4.4 
percent were issue cases. Overall, the ASAP was a far superior 
predictor than either AFQT or years of education. 


Analyses of the different content categories of biodata 
items on the ASAP indicated that issue case personnel were more 
likely to have used tobacco, had problems in school and on 
previous jobs, and committed a number of minor misbehaviors 
before entering the service. Finally, with regard to 
adjudication results for Army and Air Force personnel, ASAP 
differentiated between those personnel who were granted a top 
secret or higher clearance versus those personnel who received 
a secret clearance or were denied a clearance. 


Conclusions 


The results supported the potential use of ASAP type 
instruments as prescreens for personnel entering high security 
risk jobs. A number of caveats were identified which suggest 
that additional studies may have to be conducted using more 
refined security criteria. In addition, the operational 
implications of using prescreening instruments need to be more 
clearly delineated. Nonetheless, it appears that behavioral 
science tests such as the ASAP could potentially play an 
important role in the personnel security screening process. 
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Introduction 


Problem 


The military services have high security risk billets for 
enlisted accessions that require a top secret (TS) or sensitive 
compartmented information (SCI) clearance. These jobs are 
usually found in the communications, intelligence, cryptology, 
and electronic ratings. In addition, critical positions exist 
for new enlisted accessions in the Nuclear Weapon Personnel 
Reliability Programs run by each of the services. In order to 
be assigned to the above jobs, an intensive and costly 
background investigation must first be conducted by the Defense 


Investigative Service. The findings from this investigation 
serve as the basis for granting or denying the required 
personnel security clearance. To reduce investigative costs as 


well as to select potentially trustworthy and reliable 
personnel, each service initiates prescreening procedures prior 
to the initiation of a request for a formal background 
investigation. These procedures select out personnel 
potentially unqualified from a security perspective; the result 
is that background investigations are only requested for 
personnel successfully passing the prescreen. 


There is considerable variability across the services in 
the prescreening procedures used for applicants and recruits 
applying for high security risk jobs (Flyer, 1986). At the 
present time, it is not known which service's approach is 
working best; the procedures have not been systematically 
validated with regard to their usefulness and cost- 
effectiveness. There is a need to conduct such validation 
studies as well as to examine new procedures and approaches 
that could provide valid and cost-effective supplemental or 
alternative prescreening methods. 


Objective 


The objective of the present study was to examine the 
validity of an experimental DoD prescreening questionnaire in 
the personnel security arena. More specifically, the Armed 
Services Applicant Profile(ASAP) was investigated to determine 
if it could be used to predict the results of background 
investigations conducted for recruits applying for high 
security risk jobs. If questionnaires such as the ASAP are 
found to be valid predictors of personnel security criteria, 
they could be used as cost-effective prescreening devices to 
supplement existing prescreening approaches for high security 
risk jobs. They could also reduce the probability of assigning 
unreliable personnel to high security risk jobs. 


Background 


There has been increasing Department of Defense (DoD) 
concern with espionage incidents and other failures of security 
clearance policies to protect classified information as well 


as with the efficiency of the security clearance process. In 
response, a select panel (Department of Defense, 1982) was 
formed to review the DoD Personnel Security Program. More 


recently, a DoD security review commission (Keeping the 
Nation's Secrets, 1985) was established to review all aspects 
of DoD security practices and procedures. While a number of 
new initiatives and recommendations were generated, one area 
that received positive endorsements by both groups was 
increased research in the application of behavioral science 
tests as a part of the prescreening and investigatory 
procedures used to approve security clearances. 


The military services are not systematically using 
behavioral science tests to prescreen recruits for high 
security risk jobs. Each service has its own prescreening 
procedures, and all make use of some variant of a prescreening 
interview in combination with information from various 
questionnaires that address previous drug use, involvement with 
the law, financial problems, and other information relevant to 
suitability for assignment to sensitive jobs. Only the Air 
Force uses psychological tests (i.e., biographic, attitude, and 
personality items) as a part of its prescreening process (see 
Flyer,1986, for a more thorough discussion of service 
prescreening procedures for high security risk jobs). 


There is a need to assess whether behavioral science tests 
can validly predict personnel security criteria. If one test 
proved particularly useful, it could provide an effective 
procedure for initially prescreening personnel applying for 
high risk security jobs. Individuals successfully passing the 
first prescreen could then be screened in greater depth by the 
current programs being used by each of the services. Actual 
operational use of such a test would have to depend on a number 
of criteria, including cost benefits and the size of the 
applicant pool. However, there is first a need to assess 
whether such tests are even predictive of important personnel 
security criteria. 


Examination of the psychological test data being gathered 
by the Air Force would provide one approach for initially 
determining the validity of psychological tests in personnel 
security. However, the focus of the current effort was to look 
at an instrument that had been administered to personnel in all 
of the services. This would allow for a validity assessment 
across the different services. The Armed Services Applicant 
Profile met this requirement, and it is discussed in greater 
detail below. 


The Armed Services Applicant Profile (ASAP). The General 
Accounting Office issued a 1982 report that noted that all of 


the services were conducting research on similar biographical 
(biodata) questionnaires for screening applicants to reduce 
first term attrition (General Accounting Office, 1982). In 
response to this report and to Congressional interest in 
enlisted screening procedures that place less emphasis on high 
school graduate status, DOD initiated a study to develop and 
validate a biodata questionnaire suitable for use by all 
services as an adaptability screening instrument (Atwater, 
Walker & Weaver, 1987). The resultant biodata questionnaire 
was the Armed Services Applicant Profile (ASAP). An ASAP 
validation study against a criterion of first year attrition is 
currently being conducted by the Navy Personnel Research and 
Development Center in San Diego, California (Trent, in 
preparation). 


The ASAP could provide an ideal initial vehicle for 
examining the predictive validity of one type of behavioral 
science test, the biographical questionnaire, against 
personnel security criteria. Although the test was not 
specifically designed as a personnel security prescreening 
instrument, the ASAP includes a wide range of biographical 
items. Personal history items have been demonstrated to 
predict a variety of complex behaviors in the private sector 
(Ghiselli, 1966; Asher, 1972; Owens, 1976; Cascio, 1982). 
Also, the fact that the ASAP was designed to predict 
adaptability to military life makes it potentially relevant to 
personnel security. Personnel reliability and suitability are 
two criteria important to personnel security prescreening. 
Clearly, failure to complete the first year of military service 
due to adaptability problems indicates potential unreliability 
and unsuitability for security responsibilities in sensitive 
jobs. Specific personnel security criteria encompass a number 
of areas and these are discussed in greater detail below. 


Personnel security criteria. The basic goal of personnel 


security policy is to ensure that no person is given access to 
classified information or assigned to sensitive duties unless a 
favorable determination has first been made regarding his or 
her loyalty, trustworthiness, reliability, and judgment. For 
military enlisted accessions entering high security risk jobs, 
there is an initial cognitive screening to qualify personnel 
for training schools. In addition, service prescreening 
programs and the background investigation are used to achieve 
the above personnel security objective. 


During both the prescreening and background investigation 
phase, an attempt is made to identify adverse background 


information relevant to personnel security. This usually 
includes information on financial irresponsibility, criminal 
conduct, sexual misconduct, mental or emotional illness, 
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hostile foreign country connections, subversive activity, 
alcohol and drug abuse, and previous security violations 
(Department of Defense, 1979). The final granting of a 
security clearance affirms that an individual has no 
significant adverse background information that would make him 
or her ineligible for access to classified information. 


Clearly, the key personnel security criterion is whether 
Or not an individual demonstrates reliability, trustworthiness, 
good judgment, and loyalty in the actual handling and use of 
classified information. Failure to do so could be manifested 
at one level in excessive security violations and at the 
extreme in the deliberate compromise of classified information, 
including espionage. From a test validation perspective, 
neither of these criteria are useful. Espionage has such a low 
base rate that no existing behavioral science test could 
demonstrate significant prediction. With regard to security 
violations, the base rate is somewhat higher but still very low 
for reliable prediction; also, historical security violation 
data are not systematically maintained on centralized data 
bases and therefore cannot be accessed for large scale 
validation studies. 


For military enlisted personnel, a potential surrogate 
measure could be early discharge from high security risk jobs 
for unsuitability reasons. Flyer (1986, pp. 12-13) has 
elaborated on this criterion: 


Since a major objective of the Defense personnel 
security program is to identify people who are 
suitable for assignment to highly sensitive 
positions, later unsuitability discharge among those 
selected for these positions could be viewed in part 
as failure of the screening process. Also, and 
critically important, personnel discharged from 
highly sensitive positions for unsuitability pose a 
special security problem. A number of those 
discharged are likely to be quite bitter as a result 
of their experience during military service, many 
would be knowledgeable of sensitive equipment and 
procedures, and almost all would be facing some 
degree of financial uncertainty on their return to 
civilian life. 


For the current study, the problem with wWwsing 
unsuitability discharges as the criterion is that reliable 
measurement necessitates that individuals be tracked for one to 
four years after their entry into active service. While sucha 
longitudinal study could be undertaken in the future, there was 
a need to identify a more proximal criterion. The background 
investigation provided such a measure. 


Background investigation issue cases. An Entrance 
National Agency Check (ENTNAC) is required for all non-prior 
service enlisted accessions. In addition, personnel entering 
high security risk jobs must have the in-depth background 
investigation (BI) completed. Whereas the ENTNAC consists of a 
search of records held by various federal agencies, the BI 
includes checks of local police records, a credit check, 
educational and employment checks, interview of references, and 
in some cases, a subject interview. The Manual for Personnel 
Security Investigations (Defense Investigative Service, 1985) 
outlines the basic requirements for conducting the different 
types of BIs. In those cases where potential derogatory 
information is uncovered during the normal investigative 
procedures, an expanded investigation is often conducted in the 
problem area. This situation is called an issue case. ast: 
should be noted that the investigatory process does not pass 
judgment on the individual; rather, it merely uncovers as much 
relevant information as possible, given certain resource 
constraints. Actual review of the information for granting 
clearances occurs during a later adjudicative phase. 


Issue cases are important because they signify that there 
may be adverse information in the person's background that 
reflects on that person's trustworthiness or reliability and 
thus on his or her qualifications to hold a high level security . 
clearance. Identification of an issue case is not a perfect 
measure of whether or not an investigation yielded significant 
adverse information: 


l. When an issue case investigation is expanded, it 
sometimes turns out (perhaps 10 percent of the time) that the 
case is favorably resolved. That is, the original assessment 
that there was adverse information did not hold up when the 
specific incidents were investigated in depth. Nonetheless, 
the case still remains an issue case, since it did require an 
expanded investigation. 


2. An initial investigation may yield adverse 
information. Many investigations may not become issue cases 
even though they contain valid derogatory information. 


3. It is possible that there may be resistance to the use 
of the issue case label. The focus of the investigative 
process is to provide relevant information on different aspects 
of a person's background but not to make a judgment as to the 
severity of that information for security clearance decisions. 
The use of the issue case label may connote such a judgment and 
thus may not be used in some instances. 


4. Finally, even when an issue case contains valid 
adverse information, it may still be positively adjudicated 
during a later review process. The adjudicator may determine 


that the information is not significant enough to deny a 
security clearance. 


The above limitations might argue for focusing only on 
those small proportion of issue cases that are negatively 
adjudicated. In these instances, there would be no doubt that 
the case contained adverse information. However, despite the 
above problems, issue case status still appears to be an 
important criterion independent of the adjudication results. 
Flyer (1985) conducted analyses that pointed out the importance 
of issue case status for military enlisted accessions. He 
analyzed data for over 1.5 million non-prior service male 
enlisted personnel entering the military services during the 
period FY-74 to FY-78. About 150,000 or ten percent of this 
population had background investigations completed during their 


intial Cowr of duty. For this subsample, he determined 
unsuitability discharge rates for issue cases and non-issue 
cases as a function of high school graduation status. These 


results are reproduced in Table l. 


Enlistees whose background investigations resulted in 
issue cases were much more likely to be separated or discharged 
early for unsuitability reasons. This relationship held up 
across all services and for both high school and non-high 
school graduates. Flyer's results strongly indicate that issue 
case status is an important predictor of future unsuitability 
discharges. They also argue for using issue case status of 
background investigations as one of the criteria for the 
initial validation of potential prescreening tests. 


TABLE 1 


Unsuitability Discharge Rates by Background Investigation Results@ 


Percent Attrition Rate 
Group Army Navy Air Force 


High School Graduates 


Issue Cases 36.2 31.8 om Me 

Non-Issue Cases Weel 22.6 26.5 
Non-High School Graduates 

Issue Cases 54.9 58.8 61.4 

Non-Issue Cases 34.7 LS traf 47.2 


4Data represent all male non-prior service enlisted personnel entering 
the military services during the period FY-74 to FY-78 who had background 
investigations. Data taken from Flyer, 1985. (N = 150,000) 


Approach 


The approach used in this study was to use existing data 
from an Ongoing ASAP validation study to examine the usefulness 
of biodata items as predictors of a personnel security 
screening criterion. The prediction variables were ASAP item 
responses, along with AFQT and education, while the criterion 
variable represented the results of a background investigation 
conducted for each enlisted accession classified to enter a 
sensitive billet. 


Instrument/Predictors 


Biodata instrument - ASAP. The ASAP has two parallel 
forms with each form consisting of 130 biodata items developed 
from previous research (Atwater et al., 1987). Each form 


contains 60 common items in Part 1 that duplicate the Military 
Applicant Profile (MAP), a biodata instrument used to screen 
non-high school graduate applicants to the U. S. Army (Frank & 
Erwin, 1978). Part 2 of each form contains an additional 30 
common items plus 40 unique items. The majority of Part 2 
items are from the Recruit Background Questionnaire developed 
at the Navy Personnel Research and Development Center (Atwater 
& Abrahams, 1983). The 130 items fall into a number of 
rational content clusters, and this distribution for ASAP Form 
1 items is shown in Table 2. 


TABLE 2 


Content Clusters for Items 
on the Armed Services Applicant Profile (Form 1) 


Number Number 
Content Area of Items Content Area of Items 
Self Description jy Childhood/Family 8 

Experiences 

School Background 18 Self-Esteem 8 
Work History 16 Athletic Activity 5 
Minor Misbehaviors 1 General Perceptions y 
Social Activities 9 Alcohol/Tobacco Use 3 
Enlistment Influences ) Hobbies/Interests B} 
Service Perceptions 9 Family's Socio- 3 


Economic Status 


As can be seen in Table 2, ASAP items cover a wide range 
of background areas, with the greatest concentration in the 
general self-description, school background, and work history 
clusters. All potential ASAP items were reviewed extensively 
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to eliminate items likely to be perceived as offensive by 


applicants or biased against subgroups. In addition, highly 
intrusive items were also excluded from the final 
questionnaire. Response options to items vary as a function of 


the type of question but always include from three to five 
fixed response choices; over 70 percent of the items offer five 


response choices. Some item response options represent 
different qualitative choices while others approximate 
Likert-type scales. It should be noted that the three 


socio-economic status items, two of the school background 
items, and two of the self-description items are currently 
being excluded from scoring in the DoD ASAP study. 


Because the ASAP is currently undergoing experimental 
validation and because of its potential operational use by DoD 
in the future, specific ASAP items are not presented in this 
report. However, the following item is representative of the 
kind of self-report questions that constitute the ASAP: 


What were your grades in high school? 


A. Mostly A's 

B. Mostly A's and B's 

C. Mostly B's and C's 

BD» Mostly C's and D"s 

E. Mostly D's and below , 


Items range from those that could be potentially verified (such 
as the foregoing sample item) to more subjective and 
non-verifiable items such as “How much do you smoke?", or "How 
do you feel when people tell you what to do?". 


Other predictors. In order to determine whether the ASAP 
was a better predictor of issue cases than more traditional 
screening variables, two additional measures were included in 
the study. These included the Armed Forces Qualification Test 
(AFQT) score, a general aptitude measure used to screen all 
military applicants, and a three tier measure of education 
level (high school graduate, alternative certificate holder, 
and non-high school graduate (see Eitelberg, Laurence, Waters, 
& Perelman, 1984). The above data were obtained for all 
accessions who completed the ASAP by matching those records 
with service master tapes maintained by the Defense Manpower 
Data Center (DMDC) in Monterey, California. 


ASAP Administration 


All military applicants in the continental United States 
for the 3-month period from December 1984 to February 1985 
completed the ASAP at either a Mobile Enlistment Testing Site 
or a Military Enlistment Processing Station as a part of the 
DoD validation study. A total of 205,968 completed the ASAP, 
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which required approximately 60 minutes to administer. A 
comparison of demographic data for this 3-month sample with all 
FY 2985 applicants yielded no substantial differences with 
regard to race and gender, while some over-sampling of high 
school graduates and low AFQT scorers was evident (Atwater et 
al., in press). 


It is important to note that military applicants rather 
than new enlistees were tested in order to gain realistic 
self-responding data that included distortion to "look good" 
(Walker, 1985). Thus, empirically derived scoring keys based 
on the applicant data could control for such distortion. 


Subjects 


ASAP sample. The ongoing DoD ASAP study is tracking the 
first year attrition of all non-prior service, active duty 


accessions. For the current study, a cutoff date of 30 June 
1986 yielded a non-prior service accession sample of 55,675 
enlisted personnel. This initial sample excluded 3,490 prior 


service accessions and 73,227 non-accession applicants from a 
total active duty applicant sample of 132,492. 


As mentioned earlier, there are a number of jobs for 
incoming enlisted accessions that require a TS or SCI 
clearance. In addition, critical positions for incoming 
enlisted personnel also exist as part of the Nuclear Weapon 
Personnel Reliability Program. For the purpose of the present 
report, the above billets have been labeled as high security 
risk. jobs. They represent job categories for which the 
individual services conduct special prescreening and that 
require background investigations be completed (and positively 
adjudicated) for personnel prior to initial assignment. 


Background investigation sample. The current focus was on 


enlisted personnel applying for these high security risk jobs 
that required a DIS background investigation. Since this study 
was initiated after the recruits had successfully passed the 


prescreening procedures used by each service, there was no way 
to retroactively identify those recruits who were prescreened 
out. Selective data on all background investigations 
(including issue case status) is maintained on the Defense 
Central Index of Investigations (DCII). This data base is 
warehoused at DMDC. In order to identify ASAP accessions with 
background investigations, DMDC merged existing ASAP files from 
the ongoing DoD study with the DCII files. This merging 
yielded 3,275 enlisted personnel who had both ASAP data anda 
completed background investigation. Thus, the subjects 


selected for this study were the prescreened subsample (N 
= 3,257) of the 55,765 ASAP non-prior service accessions for 
whom a background investigation was both requested by the 
individual services and completed by the Defense Investigative 
Service. A breakout of this sample by the different services 
is presented in Table 3. 


TABLE 3 


Background Investigation Sample by Service 


Total Accessions with 
Accessions Background Investigations 
Service N N s 
Army 25,136 882 3.5 
Navy 11,963 1,001 8.5 
Air Force 12,350 1,259 10.2 
Marines 6,226 1.05 Vy 
Total 55,675 3,257 5.9 


Approximately six percent of the accessions had a formal 
background investigation. The Air Force had the largest 
proportion (10.2%) of accessions with investigations while the 
Marines had the lowest percentage (1.7%). A comparison was 
made between the demographic characteristics of the total 
accession sample and the subsample for whom a background 
investigation was completed. These data are shown in Table 4. 


TABLE 4 


Comparison of Total Accession Sample with Background Investigation 
Subsample on Selected Demographic Variables 


Percent of Total 
Accessions with 


Subgroup All Accessions Background Investigations 
CH = 55,675) CN = 3,257) 
High School Status 
No Certificate 5.4 128 
Alternative Credential 6.2 2.0 
High School Diploma 88.4 96.2 
Gender 
Female 14.0 22.0 
Male 86.0 78.0 
Race 
Black 20a 1 16:7 
White 75.0 79.5 
Other 4.9 3.8 
AF QT 
Bi (93-99) 3.5 8.8 
Bae (65-92) 31.6 45.5 
IIIa (50-64) 24.2 20.8 
IIIb (31-49) 32.6 22.4 
IV (21-30) 8.1 2.4 
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The background investigation subsample differed from the 
total accession sample with fewer minorities (20.5% vs 25.0%), 
more females (22.0% vs 14.0%), more high school graduates 
(96.2% vs 88.4%), and higher AFQT scores. The higher education 
and AFQT composition of the background investigation subsample 
reflected initial prescreening for training schools (i.e., jobs 
that require a background investigation also require a 
post-boot camp training school with aptitude cut-off scores). 


Criterion: Background Investigation Issue Cases 


Classification of a background investigation as an issue 
case Or non-issue case represented the criterion variable. 
Table 5 presents a breakout of the percentage of issues cases 


for the background investigation subsample (N = 3,257) by 
service and by selected demographic variables. 
TABLE 5 


Percentage of Issue Cases 
by Service and Selected Demographic Variables 


Number of Percentage of 

Subgroup Investigations Issue Cases 
Service 

Army 882 12.0 

Navy 19011 17.8 

Air Force 1,259 8.0 

Marines 105 76 
Gender 

Female 715 9.8 

Male 2,542 12.8 
Race 
~ Black 545 14.3 

White 2,587 Vl e7 

Otner 125 12.0 
High School Status 

No Certificate 60 31.6 

Alternative Credential 65 21.5 

High School Diploma B35 132 11:26 
AFQT 

I (93-99) 287 aS 

IT (65-92) 1,482 1064 

IIIb (50-64) 679 13.4 

IIIa (31-49) 731 14.5 

Iv (21-30) 78 17-9 


Total 37207 his. 
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Approximately 12 percent of the background investigations 
were identified as issue cases, with this rate varying from a 
high of 17.8 percent for the Navy to a low of 7.6 percent for 
the Marines. In general, the issue case rates were very 
similar across the different demographic variables. However, 
there were large differences between the percentage of issue 
cases for non-high school graduates (31.6%), alternative 
certificate holders (21.5%), and high school graduates (11.6%). 
It should be noted that over 96 percent of the total sample had 
a high school diploma. 


Adjudication. As mentioned earlier, the findings from the 
background investigations are adjudicated separately by each of 
the services prior to granting a security clearance. The 
results of this adjudication process in terms of either 
granting or denying a clearance are coded on the DCII files for 
Army and Air Force personnel. Clearance data for Navy 
personnel has just recently been added to the DCII files but 
does not include historical data relevant to the Navy ASAP 
sample. 


Although the primary focus of the current study was on the 
issue case/non-issue case criterion, it was of interest to 
determine whether the ASAP would also predict adjudication 
results. Thus, for Army and Air Force personnel only, DCII 
files were matched with the ASAP files to obtain data relevant 
to final adjudication/clearance decisions. 


Scoring Key Development 


Given the small sample size in the low criterion group 
(i.e., those personnel with issue case background 
investigations), and in order to double the N, the 90 common 
items on both Form 1 and Form 2 were treated as a single 
shortened questionnaire. Consistent with the ongoing DoD 
study, six of these common items were discarded because of 
possible bias against subgroups. The result was that the 3,257 
accessions who had background investigations were now treated 
as one sample who had completed a shortened version of the 
ASAP. All futures references to the ASAP in this report refer 
to this common core of 84 items. 


Approximately 52 percent (N = 1,679) of the background 
investigation accessions were randomly assigned to a 
scoring-key development sample; the remainder (N = 1,578) were 
assigned to a holdout sample for statistical cross-validation 
purposes. The ASAP predictor score was developed using the 
“weighted application blank" method (England, 1971). The Navy 
Personnel Research and Development Center's KEYCON program 
(Abrahams, Neumann, & Rimland, 1973) was used to select items 
for the scoring key. This was accomplished within the key 
development sample by comparing for each item choice the 
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percentage of high criterion respondents (relative to the total 
high criterion group) with the percentage of low criterion 
respondents (relative to the total low criterion group). This 
percent difference was then used as the weighted score for that 
response. 


Three different keys were developed, based on response 
differences that equaled or exceeded one, five, or ten percent. 
Using the five percent difference key as an example, if 60.3 
percent of the high criterion group in the key construction 
sample endorsed a particular response option while only 52.1 
percent of the low criterion group endorsed it, that option 
received a weight of +8.2. Similarly, if 30.3 percent of the 
high criterion group endorsed another response option while 
42.8 percent of the low criterion group endorsed it, that 
option received a weight of -12.5. For this key, percent 
differences less than five percent were scored as zero. An 
individual's total ASAP score consisted of the sum of the 
response weights for those item responses endorsed. 
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Results and Discussion 


ASAP Validities 


Table 6 presents correlations (between ASAP scores and the 
issue case criterion) from the key development and cross- 
validation samples for the three different scoring keys based 
on one percent (Key A), five percent (Key B), and ten percent 
(Key C) item response differences. These correlations were not 
corrected for indirect restriction of range of the ASAP scores 
resulting from the service security prescreening procedures; 
accurate estimates of the amount of restriction were not 
available. All point biserial correlations were corrected to 
biserial correlations because of the extreme split (90/10) on 
the dichotomous criterion. 


TABLE 6 
ASAP Validities for Different Scoring Keys 


Scoring Items Item Key Development Cross-Validation 
Key Scored Responses Sample (N = 1679) Sample (N = 1578) 
%D r r 

Pois bis Pois bis 

A (1%) 83 250 se7 wl s2e 2-36 

B (5%) 41 62 24 439 s2e «36 

C (10%) 8 10 eB: 629 sel «23d 


Note: All point biserial and biserial correlations are 
Significant (p<.001). 


The cross validation biserial correlations were all 
significant (p < .001) and were identical for Key A and Key B 
(xr = .36) and slightly lower for Key C (r = .34). Since Key B, 
based on the five percent differences, yielded an identical 
cross validity as the one percent key, yet required only 41 
scored items (as opposed to 83 items for Key A), it was used to 
further examine the relationship between the ASAP and issue 
case rates. Figure 1 graphically illustrates this relationship 
by quintiles for the total cross-validation sample. 
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Figure 1. Issue case rate by ASAP score quintiles for cross 
validation group (N=1,578). 


Figure 1 shows that almost 24 percent of those individuals 
having the lowest 20 percent of the ASAP scores had issue case 
background investigations. On the other hand, among the 
highest 20 percent of ASAP scorers, only 4.4 percent had issue 
case background investigations. Overall, the higher the ASAP 
quintile, the lower the rate of issue cases. Interestingly, 
approximately 42 percent of the issue case personnel scored in 
the lowest 20 percent quintile on the ASAP. In other words, if 
we eliminated the lowest 20 percent of the ASAP scorers, about 
42 percent of the issue cases would be eliminated. 


ASAP validities were also examined for subgroups of the 
cross-validation sample. These data are presented in Table 7 
using the five percent difference scoring key. 


Within the different services, the ASAP validities were 
Significant (p < ».001) for the Army, Navy, and Air Force 
samples and were significantly higher (p < .01) for Army (r = 
-40) and Navy (r = .38) personnel than for Air Force personnel 
(r = .22). Because of the small sample size, no significant 
results were obtained for the Marine sample. Significant and 
similar validities (r = .35 or .36, p < .001)were obtained for 
females, males, whites, and non-whites. Overall, the validity 
of the ASAP key held up well across different subgroups. 
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Content Categories 


Table 8 rank orders different content categories of 
biodata items by the degree to which they discriminated between 
issue case and non-issue case personnel. The mean item 
discrimination was computed for 13 rational content areas 
representing the 84 ASAP items. This measure, for a given 
content cluster, consisted of summing the absolute values of 
the percentage differences in item choice responding between 
the issue case and non-issue case personnel and then dividing 
by the number of items in that cluster. 


TABLE 8 


Rank Order of Content Categories 
by Mean Item Discrimination 


Content Content Number of Mean Item 

Rank Cluster Items Discrimination@ 
1 Tobacco Use 2 60.95 

2 Minor Misbehaviors 8 30.24 

3 Work History ) 22.00 

4 School Background 12 19.18 

5 Service Perceptions 1 16.57 

6 Social Activities ) 15.46 

7 Enlistment Influences 6 13.27 

8 Athletic Activity 5 12.90 

9 Hobbies/Interests 3 12.23 

10 Self-~Descriptions 13 12.01 

11 Childhood/Family Experiences 6 11.95 

12 General Perceptions 1 11.90 

13 Self-Esteem 3 7-23 
-- Total Scored Items 84 18.91 


4 This measure represents the absolute value of the percentage 
differences between the issue case and non-issue case groups summed across 
response choices in a content cluster and then divided by the number of 
items in that cluster. 


By far, the content cluster that showed the greatest 
discrimination addressed tobacco use (the alcohol use item that 
was Originally part of this cluster was not one of the 84 ASAP 
items common across the two forms). The second best 
discriminator was minor misbehaviors, followed by work history 
and school background. The other nine categories ranged from a 
high of 16.57 for items on service perceptions to a low of 7.23 
for self-esteem items. 


In order to provide more concrete examples of the types of 
items that discriminated between the issue case and non-issue 
case personnel, the same analytical technique was used to rank 
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order the 15 items showing the greatest between-group 
discrimination. These results are shown in Table 9. 


TABLE 9 


ASAP Items Showing Greatest Between-Group Discrimination 


Issue Case Personnel Content Total Item 

Rank More Likely to: Cluster Discrimination@ 
1 Smo ke 12 61.60 
2 Start smoking at younger age 12 60.30 
3 Be suspended from school at 

younger age 4 43.80 
4 Have a tattoo 4 43.60 
5 Be fired from a civilian job 3 43.60 
6 Be expelled from school 4 42.60 
He Have thought about quitting 

school 2 30.80 
8 Have parents who were unhappy 

about their school grades 2 30.40 
9 Have lower grades in school 2 29.40 
10 Stay out more on weekends 5 27-40 
11 Have been in trouble with 

police 4 26.00 
12 Have had more full time jobs 3 25.30 
13 Not plan on making service a 

career é 25.30 
14 Have school grades lower than 

they expected 2 24.00 
15 Have applied for more jobs 3 23.30 


4 This measure represents the absolute value of the percentage 
difference between the issue case and non-issue case groups summed across 
the response choices for an item. 


Those personnel whose background investigations resulted 
in issue cases were more likely to smoke and to have started 
smoking at a younger age. They were more likely (when compared 
to non-issue case personnel) to have (1) made lower grades than 
either they or their parents expected, (2) made lower grades 
than non-issue case personnel, (3) experienced problems in 
school, including having been expelled and/or suspended and 
having thought about quitting school, (4) held and/or applied 
for more full time jobs, (5) been fired from a civilian job, 
(6) gotten a tattoo, (7) stayed out more on weekends, and (8) 
decided not to make the service a career. This profile 
suggests that the typical individual with an issue case 
background investigation had turned away from an unhappy school 
situation to seek an outside job and socio-economic rewards. 
The fact that the individual had problems both in school (e.g., 
more likely to have been expelled or suspended) and on the job 
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(e.g., more likely to have been fired) suggests a general 
factor of unreliability. Overall, this profile does not 
represent the ideal type of person that the military services 
would want to assign to a high security risk job. 
Surprisingly, all of these individuals had successfully passed 
the service prescreening procedures since they all had a 
background investigation. 


Other Predictors 


In order to assess whether the ASAP was a more valid 
predictor of issue cases than other commonly used measures, 
AFQT scores and years of education were examined. Table 10 
presents the results of the analyses assessing the relationship 
for personnel in the cross-validation sample between these 
predictors and the issue case criterion. 


TABLE 10 


Relationship Between Predictors and Issue Case Criterion 
(Cross-Validation Sample N = 1,578) 


Mean Standard Deviation 
Issue Non-Issue Issue Non-Issue r ie 
Variable Case Case Case Case Phis bis 
ASAP 37.0 1563 54.5 56.4 ooo te 2 36** 
AFQT 62.1 66.4 19.4 19.4 o7" 0088 
Years Education Ti2 a 1.0 1.2 06" .10## 


Note: Issue case criterion is 1 = issue case and 2 = non-issue case. 
Years of education ranges from 1 = 1-7 years to 13 = doctorate or 
equivalent. 


*# p<.01 
#® p< .001 


AFQT scores and years of education were significantly 
correlated with the issue case criterion (biserial r's of .11l 
and .10, respectively); however, these relationships were not 


as strong as that for the ASAP (biserial r = .36). Non-issue 
case personnel scored considerably higher on the ASAP (mean = 
75.3) than their issue case counterparts (mean = 37.0). 


Regression analyses were undertaken to examine whether the 
combination of the three predictors would improve prediction. 
In the first regression analysis, AFQT was forced into the 
stepwise regression equation first, followed by years of 
education and ASAP. In the second regression analysis, the 
variables entered in their natural order (ASAP, AFQT, then 
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years of education). The results of these analyses are shown 
in Table ll. 


TABLE 11 


ASAP Incremental Validity 
(Cross-Validation Sample N = 1,578) 


Step Simple Multiple Significance 
Variable ia R of Change 

p< 
(1) AFQT elit 2114 -001 
(2) Years Education «iO 132 NS 
(3) ASAP - 36 -362 001 
(1) ASAP - 36 - 356 001 
(2) AFQT oft - 360 NS 
(3) Years Education -10 - 362 NS 


Note: All correlations have been converted from point-biserial to biserial. 


While the ASAP significantly added to the prediction when 
AFQT and years of education were entered first, neither AFQT 
nor years of education significantly added to the prediction 
when ASAP entered first. Indeed, the prediction using ASAP 


alone (biserial r = .358) was almost identical to the 
prediction obtained when using all three variables in 
combination (biserial r = .367). Overall, the ASAP was a far 


superior predictor to either of the other two measures. 


Adjudication Results 


Table 12 presents the results from the adjudication phase 
for Army and Air Force personnel, broken down by the issue case 
status of the background investigation. 

TABLE 12 


Adjudication Decision by Issue Case Status 


Background Investigation 


Clearance Issue Case Non-Issue Case Total 
nN & N Z N t 
Denied 7 (33) 14 (67) 21 (1) 
Pending 
Adjudication 31 (29) TT (71) 108 (5) 
Secret i" (12) 124 (88) 141 (7) 
Top Secret/SCI 147 (8) 1684 (92) 1831 (87) 


Note: Table includes only Army and Air Force personnel. 
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Only one percent of the sample had their clearance denied 


based on information from the background investigation. Five 
percent of the sample had the adjudication decision still 
pending. This category often represents personnel for whom the 


adjudicator has requested additional information either because 
the overall investigative results are incomplete or, more 
likely, because adverse information needs to be documented in 
greater detail to potentially support a clearance denial. The 
seven percent of the sample in the secret category could either 
represent individuals whose new job turned out not to require a 
top secret/SCI clearance, or, in some cases, individuals whose 
background investigation yielded results that would disqualify 
them from the highest level clearances but would still support 
a secret clearance. Eighty-seven percent of the sample 
received a top secret/SCI clearance. 


An examination of Table 12 with regard to issue case 
status of the background investigation provides some 
interesting findings. Two-thirds of those personnel who had 
their clearance denied did not have issue case background 
investigations. Likewise, over 70 percent of the personnel in 
the pending adjudication category were not issue cases. These 
results substantiate earlier assertions that a large number 
(perhaps as high as 2/3) of the personnel with adverse 
background information are not classified as issue cases. 


A comparison of the ASAP scores for personnel in each of 
the four adjudication categories is presented in Table 13. 


TABLE 13 


Comparison of ASAP Scores 
by Different Adjudication Categories 


ASAP Scores@ 


Clearance N Mean 
Denied 21 38.8 
Pending 

Adjudication 108 47.9 
Secret 144 S703 
Top Secret/SCI 1,831 78.8 


ar = 20.2, Pp < -001, (df = ‘3% 2097) 
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Conclusions 


The objective of the present study was to conduct an 
exploratory study examining the validity of behavioral science 
tests in predicting personnel security criteria. More 
specifically. an assessment was made to determine the degree to 
which the ASAP could predict the issue case status of 
background investigations. The results were very encouraging. 
ASAP scores significantly predicted the issue case criterion 
and also differentiated between those Army and Air Force 
personnel who were granted a top secret/SCI clearance versus 
those personnel who received a lower clearance or were denied a 
clearance. While the results support the potential use of 
ASAP-type instruments as prescreens for personnel entering high 
security risk jobs, a number of caveats emerge. 


First, the present study could not identify those 
personnel who were prescreened out by the service prior to the 
formal request for a background investigation. This selected- 
out group is especially important because the greatest 
feasibility in using instruments such as the ASAP may be 
identifying those personnel who have a high probability of 
being prescreened out by the services. Administration of an 
instrument such as the ASAP may be far more cost effective than 
conducting more in-depth prescreening on a total applicant 
pool. Future research must determine whether or not 
instruments such as the ASAP can cost-effectively perform this 
function. 


A second limitation of the current study concerned the use 
of issue cases as the key criterion. The fact that two-thirds 
of the personnel who were denied clearances were not issue 
cases indicates that there is a need to better measure the 
degree to which background investigations contain adverse 
information. It is certainly too costly and time consuming for 
researchers to review and classify hard copy background data 
for the large samples usually found in test validation studies. 
One alternative may be to have adjudicators provide ratings of 
the degree of severity of the adverse background information in 
those cases where they do not deny a clearance. These ratings 
could be done both for the overall investigation and by 
different content areas (e.g., involvement with law, substance 
abuse, financial problems, etc.). Such ratings could then be 
used for research purposes only. Clearly, the degree to which 
policy makers will accept the implementation of prescreening 
tests will heavily depend on the reliability and validity of 
both these tests and the personnel security criteria used to 
validate them. 
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Lastly, and perhaps most importantly, the present study 
was conducted as an exploratory effort. The recruiting, 
classification, and other operational implications of 
implementing instruments such as the ASAP were not examined in 
depth. There is clearly a need for a thorough and detailed 
mapping of the prescreening procedures and policies used by 
each of the different services. 


Flyer (1986) has provided initial data for this mapping. 
However, such a study should expand on Flyer's work to include 
detailed information for each of the services on different 
Organizational responsibilities, different information 
collected, and different decision processes. Such information, 
properly organized, could provide a framework for identifying 
those points in the prescreening process where tests such as 
the ASAP could provide the most leverage. It could also 
identify interaction points across the services that might 
provide opportunities for cost savings through improved 
consolidation. 


Despite the above caveats, the findings from the current 
study are encouraging. It appears that behavior science tests 
such as the ASAP can potentially play an important role in the 
prescreening of applicants for high security risk jobs. 
Implementation issues are critical but are not insurmountable. 
Future studies in this area could provide the operational 
detail necessary to firmly implement the recommendations of the 
two recent security panels/commissions -- that behavioral 
science tests, if valid, be incorporated into the prescreening 
procedures used to approve security clearances. 
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